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Introduction 

Recent survey on employed international students at different institutions at the 
U.S. (The Chronicle of Higher Education, Nov. 28. 1990) found that at 176 Ph. D. 
granting institutions there are 46,479 international scholars. Thirteen percent of 
international scholars are occupied with teaching at these institutions. Prior -f« 
coming to the U.S. thpse international students are granted either a teaching or 
research assistant position. In most institutions granting a teaching assistantship or 
research assistantship to a new international graduate student works as a contract 
between universities and these individuals, an arrangement which guarantees 
financial assistance to these incoming international graduate students who are 
admitted to different degree programs. By state law or university regulations they 
may be required to prove that they have sufficient oral English proficiency in order 
to be able to carry out instruction in an American institution. Frequently they are 
required to take the SPEAK test ( The SoeakinQ Proficiency English Assessment 
Kill which is the institutional version of the Test of Spoken English , developed by 
the Educational Testing Services (ETS). (1982) 

The SPEAK Test has seven sections. Each of the sections is timed. The first 
section is not scored because it is used for a warm-up; it includes some warm-up 
types of questions to make the ..minee feel at ease. In the second section the 
examinee is asked to read a paragraph loud in a minute. In the third secrion \he 
examinee is given ten incomplete written sentences to be completed orally. The 
picture story section follows as the fourth section: the examinee is given a series of 
pictures to examine and is asked to tell a story. In section five the examinee 
studies a picture and answers four questions related it. In section six there are 
open-ended questions. In the last section examinee studies a class schedule 



and then explains this it. The topics covered in this test are general, everyday 
topics which have very little to do with most ITAs' (International Teaching Assistant) 
majors. 

Most of the ITAs, roughly speaking, 60 %, are employed in the fields of science 
and technology. ITA education and training is a field which has not been given 
enough emphasis; nor have there been clear cut decisions made by governments 
and institutions. Goodman and Nacht (1983) view the education of foreign 
students in American colleges and universities as an "absence of decision" . 
Recruiting ITAs to teach introductory courses is not an exception to this 
phenomenon. So far most institutions have been usirg the SPEAK Test as a 
screening device without much scrutiny. 

Some descriptive studies that have been done in this area have criticized the 
SPEAK Test. Barrett (1987) and Braswell and Green (1987) have discussed the 
inadequacies of this test as not being a valid and reliable testing instrument . It is 
also a common belief that most ITAs who have taken this test have not had good 
impressions of it. The purpose of this paper is to see how the SPEAK Test is 
perceived by both ITAs and instructors at a major midwestern university where it is 
used as a screening instrument for hiring ITAs. 

In fact, this paper has stemmed from two areas of inquiry. The first is research 
with international teaching assistants. Within this broad area there have been 
several studies published on the SPEAK. Earlier studies on the SPEAK Test 
concentrated on validity and reliability issues. Abraham & Plakans (1988) 
provided an overall evaluation of the SPEAK and in-house TEACH tests. In 
addition, there are some studies that compare the SPEAK Test with other ways of 



assessing oral proficiency, such as the FSI oral interview (Mellor, 1987) and field- 
specific tests (Smith, 1989). Clark and Swinton (1980) found that both the Test of 
Spoken English (TSE) and the FSI (Foreign Service Institute) Oral Interview are 
valid and effective predictors of oral proficiency. The FSI seems to have slightly 
higher face validity. Bejar (1985) and Stansfield, and Ballard (1984) have also 
done research on validity and reliability issues. 

The other major area of research in second language testing is affective 
reactions of test takers to different kinds of language tests (Savignon, 1972; 
Brutsch, 1979; Jones et al. 1980; Shohamy, 1982; Scott and Madsen, 1983; 
Madsen, 1982; Madsen and Murray, 1984.) Various factors, such as length, format, 
limited time for giving answers, whether the test is perceived as a valid test or not, 
clarity of directions to the test, and anxiety were found to influence students* 
perceptions of tests that they take. Spielberger (1966) o.otinguished between trait 
anxiety (a stable personality characteristic) and state anxiety (transitory). 
Schwarzer, van der Ploeg, and Spielberger (1982) argued that state anxiety is 
seen more frequently in people who have higher instances of trait anxiety. Alpert 
and Haber (1960) talk about a facilitating and/or a debilitating effect of anxiety on 
the end result. Madsen (1982) and Hill (1983) believe that state anxiety causes 
more debilitating effect on people. Hill (1983) pointed out the negative effect of 
time limitations on individuals. As a result of his study children who had high test 
anxiety made errors three times more often than the children who were considered 
as having low test anxiety. Madsen's (1982) study on ESL students showed that 
several factors such as unclear directions, lack of face validity, insufficient time, 
format of the exam (cloze, multiple choice, interview, etc) seem to create anxiety in 
people. 



An interesting finding by McKeachie et al. (1955) and Calvin et al. (1957) is that 
to be able to talk and express how they feel about a test releases anxiety. The 
Savignon (1972) and Shohamy (1980) studies pointed out that students have a 
pleasant and relaxing experience when they take an oral interview. Even if they 
did not show great performance during the interview they felt the interview was a 
fair and valid type of test; it tested what they should be asked to perform in a real 
life situation. 

Scott (1980) in her study on Japanese and Spanish test takers revealed that 
there was a difference in affective reactions towards certain items that involves the 
individuals emotionally in a crosscultural perspective. Spanish speakers seemed 
to react to the items that frustrate them more positively than the Japanese. 

Method 
Subjects 

The data analyzed in this study were collected in the Summer quarter of the 
1 989-1 990 academic year from 24 ITAs who took the SPEAK Test in 1 988-89 and 
24 ITAs who took the same test in 1989-90 academic years, thus totalling 48 
volunteer ITAs. Twelve of the 24 ITAs in each group were from the social sciences 
and the other half (12) were from math-based sciences. Different engineering 
fields, math, physics, chemistry, genetics, astronomy, computer science, forest 
products, biochemistry, food science and nutrition, and economics are all 
considered math-based sciences. On the other hand, psychology, sociology, 
English linguistics, foreign languages (French, German, etc.), journalism. East 
Asian studies, education, anthropology, art history, and geography represent the 
social based sciences included in this study. 
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The volunteer ITAs for this study were from the People's Republic of China 
(PRC-15 individuals). Korea(IO). Taiwan (4). Hong Kong (2). Japan (2), Panama 
(1). Peru (1 ). India (3). Iran (1 ). Turkey (3). U. S. S. R. (1 ). Israel (1 ). Nigeria (1 ). 
Finland (1), and Vietnam (1). For part of the statistical analysis three major groups 
were formed. The first consisted of ITAs from the PRC, the second major group 
had ITAs from Korea, and ITAs from Taiwan, Hong Kong, and Japan constituted 
the third distinct group. 

Volunteer ITAs in the study: 



PRC 


15 


Korea 


10 


Taiwan 


4 


Hong Kong 


2 


Japan 


2 


Panama 


1 


Peru 


1 


India 


3 


Iran 


1 


Turkey 


3 


U.S.S.R 


1 


Israel 


1 


Nigeria 


1 


F-.,,.ind 


1 


Vietnam 


1 



Procedure 

The researcher prepared an affective reactions questionnaire of total 32 items. 
(See Appendix I) This questionnaire was given to 24 ITAs who took the SPEAK 
Test in the '88-'89 academic year, and to 24 other ITAs both in math and social 
science fields who took the SPEAK between January and June of 1 990. The first 



part of the questionnaire elicited biographical data from the subjects. Questions # 
3, 4, 31 and 34 required quantitative information. The format of the rest of the 
questions was organized as a four-point Likert-styie scale. All the items have 
space for the ITAs to explain their ratings. Some of the subjects wrote in their 
reactions after the rating; some had an oral follow-up interview. 

Oral interviews with some of the ITA program instructors were also conducted 
individually and recorded. 

Hypotheses (questions) 

In this study the following questions were asked to investigate the reactions of 
ITAs to the SPEAK Test: 

1. Is there a difference between what math-based science (ITAs) and ITAs coming 
from the social sciences think about the SPEAK Test? 

2. Is there a difference between the attitudes toward the SPEAK Test of ITAs who 
took the SPEAK Test during 1988-1989 and during the 1989-1990 academic year? 

3. What is the relationship of cultural (ethnic origin) background of ITAs to how well 
they perceive the SPEAK Test? 

4. Is there an attitude difference toward the SPEAK Test between ITAs who passed 
the SPEAK test and those who did not? 

5. What is the relationship of gender and attitudes toward on the SPEAK Test? 

6. Is age related to attitudes toward the SPEAK Test? 
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7. What is the relationship between passing the SPEAK Test and the amount of 
English ITAs have hac!? 

Analysis 

After the questionnaires were collected. 21 items out of 32 were chosen for 
analysis in this study. The remaining 1 1 questions were specific questions that 
were not relevant to every ITA. Based on these 21 items a total reaction score was 
calculated for 48 ITA's who participated in the study. There were four blanks in the 
questionnaire and values, 1 to 4 were attributed to the blanks. The most positive 
got a score of 4 and the least positive got a score of 1 . Overall the total attitude 
score towards the SPEAK Test is rather low; scores ranged from 37 to 71 out of 84. 

Question # 1 & 2: 

Hypothesis 1: There is a difference between what math-based ITAs and social 
science-based ITAs think about the SPEAK Test. 

t^ypotiiesis 2: There is a difference between the attitudes toward the SPEAK Test 
of ITAs who took the SPEAK during 88-89 and during 89-90 academic years. 

A two-way anova was computed to see if there is any significant difference in 
total attitude score between groups on these W^o factors. The means of the groups 
on the factors are very close to each other and they do not show a statistically 
significant difference neither between year nor across fields. Then a two-way 
anova by year (88-89, 89-90) and field (math, social science) for each item was 
done. Some of the items showed a significant difference in means. Questions 6, 
12, and 27 proved to be statistically significant in the analysis when the ITAs of 
1989-90 were compared to the ITAs who took the SPEAK Test last year. On the 
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other hand, questions 13, 25, and 29 were perceived differently by ITAs that came 
from different backgrounds. In the interaction of year and field questions 6 and 26 
were found to yield significant interactions. Results are presented in Tables 1,2,3, 
and 4. 

TABLE I 

TWO WAY ANOVA ON TOTAL ATTITUDE SCORE BY YEAR AND FIELD 



Means 


Malh'Sd 




SocSci 






Year 1 


50.92 




51.75 


51.33 




Year 11 


57.00 




52.00 


54.50 




Total 


53.9'3 




51.88 






ANOVA TABLE 










sy 


m 




MS 




£ £ 


Year 


1 


120.33 


120.33 




2.369 .131 


Field 


1 


52.1 


52.1 




1.02 .317 


YearXFieid 


1 


102.1 


102.1 




.163 


Residual 


44 


2235.2 
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TABLE 2 

Summary of Two-way Anovas On Each Item of the SPEAK Test (by Field) 
Means 



Items 


Math'Sci 




F 


P 


Q1 


5.00 


5.00 


0.00 


1 00 

\ ,\J\J 


Q2 


3.833 


3.584 


0.25 


0 fil 


Q5 


4.250 


5.417 


3.24 


0 OR 


Q6 


6.666 


6.167 


0.77 


0.34 


Q8 


5.333 


4.750 


0.87 


0.35 


Q9 


6.333 


6.666 


0.25 


0.61 


Q11 


5.834 


6.166 


0.29 


0.59 


Q12 


5.167 


5.667 


0.56 


0.45 


Q13 


4.417 


3.417 


4.28 


0.04' 


Q14 


4.00 


2.917 


2.95 


0.09 


Q15 


4.250 


4.250 


0.00 


1 ,00 


Q17 


3.583 


3.416 


0.12 


0.72 


Q19 


6.500 


6, SCO 


0.00 


1 .00 




O . OO'I 






U.d4 


Q21 


7.166 


7.250 


0.04 


0.83 


Q22 


4.250 


3.666 


1.15 


0.28 


Q25 


7.750 


6.917 


3.79 


0.05* 


Q26 


3.916 


4.833 


0.30 


0.58 


Q27 


4.333 


4.416 


0.01 


0.90 


Q28 


4.416 


3.916 


0.53 


0.4/ 


Q29 


5.417 


4.000 


4.04 


0.05" 



* SignHicant at .05 
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TABLE 3 

Summary of Two-way Anovas On Each Item Of The SPEAK Test (by Year) 

Means 



Items 


Year1(88-89^ 


Year 2 (89-901 


£ 


E 


Q1 


4.500 


5.500 


3.09 


0.86 


Q2 


3.250 


4.167 


3.38 


0.07 


Q5 


4.834 


4.834 


0.00 


1 .00 


Q6 


5.833 


7.000 


4.17 


0.04* 


Q8 


4.916 


5.167 


0.16 


0.69 


Q9 


6.666 


6.333 


0.25 


0.61 


Q11 


6.500 


5.500 


2 65 


0.11 


Q12 


4.667 


6.167 


5.09 


0.02* 


Q13 


3.834 


4.00 0 


0.11 


0.73 


Q14 


3.167 


3.750 


0.85 


0.36 


Q15 


3.917 


4.583 


1.22 


0.27 


Q17 


3.583 


3.416 


0.12 


0.72 


Q19 


6.666 


6.334 






Q20 


5.334 


5.500 


0.05 


0.81 


Q21 


7.416 


7.000 


1.04 


0.31 


Q22 


3.750 


4.166 


0.58 


0.40 


Q25 


7.167 


7.500 


0.60 


0.44 


Q26 


4 083 


4.083 


0.00 


1.00 


Q27 


3.666 


5.083 


4.50 


0.03* 


Q28 


4.166 


4.166 


0.00 


1.00 


Q29 


4.667 


4.750 


0.01 


0.90 



* Significant at .05 
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TAbLE 4 

Summary of Two-way Anova on Each Item of the SPEAK Test(Year by Field Interaction) 

Means 



Hems 


Math 1/ Yean 


Math PA'ear 2 


SoScil/Ypnr 1 


SoSci2/YGar 2 


£ 


£ 


Q1 


2.167 


2.833 


2.333 


2.667 


0.34 


0.56 


Q2 


1.583 


2.250 


1.667 


1.917 


0.69 


0.40 


Q5 


2.333 


1.917 


2.500 


2.917 


1.65 


0.20 


Q6 


3.333 


3.333 


2.500 


3.667 


4.17 


0.04* 


Q8 


2.333 


3.000 


2.583 


2.167 


3.02 


0.08 


Q9 


3.083 


3.250 


3.583 


3.083 


1.02 


0.31 


Q11 


3.417 


2.417 


3.08:^ 


3.083 


2.65 


0.11 


Q12 


2.000 


3.167 


2.667 


3.000 


1.57 


0.2: 


013 


2.167 


2.250 


1.667 


1.750 


0.00 


1.00 


Q14 


1.750 


2.250 


1.417 


1.500 


0.43 


0.51 


Q15 


1.917 


2.333 


2.000 


2.250 


0.07 


0.78 


Q17 


1.750 


1.833 


1.833 


1.583 


0.51 


0.47 


Q19 


3.333 


3.167 


3.333 


3.167 


0.00 


1.00 


Q20 


2.667 


2.91" 


2.667 


2.583 


0.21 


0.64 


Q21 


3.833 


3.333 


3.583 


3.667 


2.04 


0.15 


Q22 


1.9;7 


2.333 


1.833 


1.833 


0.58 


0.44 


Q25 


3.750 


4.000 


3.417 


3.500 


0.15 


0.69 


Q26 


1.583 


2.333 


2.500 


1.750 


6.08 


o.or 


Q27 


1.583 


2.750 


2.083 


2.333 


1.88 


0.17 


Q28 


2.083 


2.333 


2.083 


1.833 


0.53 


0.47 


Q29 


2.417 


3.000 


2.250 


1.750 


2.36 


0.13 



" significant at .05 
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Question # 3 

Hypothesis 3: There would be a difference in th^perception of the SPEAK Test 
among three different Asian groups. (PRC- Korea-Taiwan, Hong Kong, Japan). 
There does not seem to be a difference whether ITAs are from Japan or Korea or 
China, they all perceive the test in the same way. The People's Republic of China 
has 15 ITAs, Korea alone has 10 ITAs, and lastly Japan, Taiwan, and Hong Kong 
have 8 ITAs in the cross-cultural comparison. The results of this analysis are 
reported in Table 5. The means of these groups in terms of their total attitude 
score are very close to each other. The null hypothesis of equal mean:, was not 
rejected. 

Other one-way anovas reveal that there is no difference among these three 
Asian groups with respect to their mean ages or the years of formal English that 
they have had back in their countries. (Refer to Tables 6 & 7). 

TABLE 5 

COMPARISON OF TOTAL ATTITUDE SCORE MEANS FOR ASIAN GROUPS 
Chinese (N=15) Korean(N=lO) Japanese-Hono Kona-Taiwan [N=8) 

Means 51.40 51.20 55.50 



ONE-WAY ANOVA ON TOTAL ATTITUDE SCORE 





a 




Kg 


f P 


Group 


2 


106.13 


53.06 


1.07 0.35 


Residual 


30 


1475.20 


49.17 




Total 


32 


1581.33 
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TABLE 6 

COMPARISON OF AGE MEANS FOR ASIAN GROUPS 

Chine se Korean Japanese-Hong Kono-Taiwan 

Means 34.7 29.8 27.8 

ONE-WAY ANOVA ON AGE 
SV d[ ^ EE 

Group 2 298.14 149.07 2.88 0.07 

Residual 30 15552.03 51.73 

Total 32 1850.18 57.81 



TABLE 7 

COMPARISON OF YEARS OF FORMAL ENGLISH MEANS FOR ASIAN 

GROUPS 

ph'^^se Korean Jaoanese-Hong Kong-Taiwan 

Means 7.87 9.10 8.38 



ONE-WAY ANOVA ON YEARS OF FORMAL ENGLISH 





df 


as 


MS 


F 


P 


Group 


2 


9.12 


4.56 


0.62 


0.54 


Residual 


30 


218.50 


7.28 






Total 


32 


227.63 
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Hypothesis 4: There is an attitude difference toward the SPEAK Test between ITAs 
who passed the SPEAK test and those who did not. A one-way anova with two 
groups who computed to see if there is a significant difference in attitude scores 
between ITAs who have scored enough to be considered for teaching- and those 
who did not. Individuals who passed the test seem to think more highly of the test 
than those who did not. Results are presented in Table 8. 

TABLE 8 

COMPARISON OF PASS/FAIL GROUPS ON TOTAL ATTITUDE SCORE 

PASS FAIL 



Means 55.68 51.10 

ONE-V^AY ANOVA ON TEST OUTCOME 

SV S 1^ f P 

P/F 1 240.87 240.87 4.88 .03* 

Residual 46 2268.79 49.32 

Total 47 2509.66 



* significant at .05 

Hypothesis 5: Male ITAs and female ITAs in this university think about the SPEAK 
Test differently. A one-way anova comparison of male and female ITAs showed 
that there is no significant difference in mean attitude score for male and female 
ITAs (See Table 9.) 

TABLE 9 

COMPARISON OF MALE AND FEMALE GROUPS ON TOTAL ATTITUDE SCORE 
MALE FEMALE 
Means 51.36 55.10 

ONE-WAY ANOVA ON ATTITUDE SCORES OF MALE AND FEMALE iTAS 



21 


a 






F 


P 


M/F 


1 


163.43 


163.43 


3.2 


0.08 


Residual 


46 


2346.22 


51.00 






Total 


47 


2509.66 
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Hypotheses 6 and 7: Age and years of tornnal English will make a difference in the 
way ITAs think about the SPEAK test. Results presented in Table 10 show that age 
is not an important factor influencing how these ITAs think about the test. 
However, fewer years of English does appear to correlate with higher attitude 
scores on the SPEAK test. 



TABLE 10 

CORRELATION AMONG ATTITUDE, AGE & YEARS OF FORMAL ENGLISH 

X3 X4 X5 

X3 1.00 -.05 -.49 X3 total attitude score 

X4 -.05 1.00 -.09 X4: age 

>S --49 -.09 1.00 X5: years of formal English 



Discussion 



The SPEAK Test is perceived by international teaching assistants to be one of 
the steps in the procedure for applying to a higher education institution in the U.S. 
Most of the oral interviews with these ITAs revealed that they perceive the SPEAK 
Test as a major test. Both male and female students seem to have the same 
feeling. They see this as one of those tests that needs to be taken in order to be 
accepted to a graduate degree program. One of the social science ITAs believes 
that it is not a very accurate evaluation of anybody's ability, and that it does not 
require any analytical ability. 

When we examine the overall reaction of both math-based and social science- 
based ITAs there is no difference in their perceptions. Howe\'er, item-by-item 
statistical analysis revealed some differences in reaction to some questions. 

1 6 



Interaction between field and year in the statistical analysis showed that most 
people think that the SPEAK Test alone is not enough of a criterion to tell about 
their oral proficiency. They wished they had had been tested with different kinds of 
tests besides the SPEAK. Social science-based ITAs have slightly higher means 
(4.250) than the ITAs that belong to science and engineering (3.916). In addition, 
math-based and social science-based ITAs think differently about question # 13, 
which asks them whether they were content with the answers that they had given to 
the questions on the test. Individuals that belong to math-based sciences 
appeared to be satisfied with the answers that they had provided during the limited 
time. This is perhaps because social scientists tend to rely mostly on verbal 
explanation, while individuals in science and technology tend to explain most of 
their data in numerical and graphic forms, and in scientific prose instead. Thus 
they did not express as much dissatisfaction as their counterparts among social 
science ITAs. However, math-based ITAs think positively about the idea of one of 
the professors interviewing them instead of taking the SPEAK test, and they favor 
this idea more than social science ITAs do. One might assume that in engineering 
fields the questions that professors ask may come across as technical, not 
needing too much verbal explanation. 

Dissatisfaction shown in the content of the answers given after each question is 
due to two major factors. Almost all the interviewees complained about the limited 
time supplied at the end of each question for test takers; the time allocated for each 
item does not exceed one minute. An educated university-level native or nonnative 
speaker of a language, in order to be content with the test questions, wants to give 
some interesting, well-thought-out answers to open-ended questions. The second 
reason may stem from the level of some questions in the test. One of the ITAs 
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feels that there should be more sensible topics to talk on. She thinks the test 
needs a lot of improvement: 

"1 felt the SPEAK test was so simple. God, this is such an insult to me. 
Sometimes the topics were so foolish, and I did not want to talk about them. This is 
most humiliating, when you are talking you are not talking sense." 

She wished she had taken the test jointly with a group discussing a topic of 
interest. She believes that the SPEAK test by itself is not a good test, but if it can be 
combined with other tests, it may provide a sufficient way of testing nonnative 
speakers for classroom corr.munication. 

One of the individuals who passed the test believes that the pictures in the test 
were so funny that she enjoyed taking the test. Yet she was very dissatisfied with 
the answers she gave to the pictures; she felt that whatever she did in the test she 
rushed over. One lacks a real situation and interaction in a real classroom 
situation. The lack of testing sociolinguistic and strategic competence in 
performing the tasks is very obvious to the test-takers as well as to test-raters. 

During interviews mostly math-based ITAs pointed out the significance of being 
able to use general English in an American classroom. They know that they can 
not operate on the technical register all the time. American undergraduates' 
questions in the classroom vary from subject-specific questions to questions 
related to classroorr.-and course-specific routines. 

The next interesting difference between these two groups of fields is the 
reaction elicited for item number 25 which is about the accent of the speaker: 
Math-based ITAs are not as comfortable as social science ITAs in understanding 
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the accent of the speaker on the tape. Teaching assistants in math-related fields 
may not be exposed to narrative type of speech as much as social science-based 
ITAs. 

When we look at the differences in perceptions between ITAs who took the 
SPEAK test at different times, the results show that ITAs of the 89-90 academic year 
are more critical about the SPEAK test. They do not believe that they were tested 
for their oral proficiency as much as the ITAs who took the test a year earlier. ITAs 
who took the test during the 89-90 academic year seem to have more positive 
feelings about the idea of taking an oral exam in a group, debating, or talking to 
another person, rather than ;alking to a tape. However, ITAs who had taken the test 
earlier did not consider this idea as highly. 

It seems that ITAs in social sciences are in search of a new test instead of the 
SPEAK test. The interaction between different fields and different years indicates 
that social-science ITAs would rather have different kinds of tests, such as the 
TOEFL, an interview with a professor, or exposure to a teaching situation, all 
affecting the department's decision to hire them as ITAs. At the same time they feel 
that these alternative ways of testing may be personally and academically more 
satisfying. 

The results of this study show that other than the test outcome (Pass or Fail - 230 
points for the University of Minnesota) other variables such as age, gender, 
different majors or cultural backgrounds do not seem to make significant 
contributions to the way ITAs think about the SPEAK test. The only interesting 
finding concerning their background is the level of their English. Individuals with 
fewer years of formal English education seem to think more positively aboui the 
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SPEAK Test. As the proficiency level goes up the individuals seem to become 
more critical about the SPEAK. Individuals with fewer years of English may have 
naive ideas about testing. 

Interviews with ITA instructors: What do they think about the test? 

Interviews with five different instructors at the University of Minnesota pointed 
out the following thoughts about the SPEAK Test. 

Face Validity: The test does not seem to test teaching skills that ITAs need to 
perform. When someone gets a high score, the TA English program would not be 
able to tell if this individual would be able to teach or not. The test may be useful in 
separating out those individuals who have language problems. So it may be 
simply a test of language. It does not seem to be useful as an actual indicator of 
classroom performance in a field-specific area. One of the ITAs said that he was 
expecting to speak, but to speak about his major. There is nothing motivating 
them to talk. No one is asking questions, they do not know what direction to take. 
This test does not test their teaching skills, but it is a test of English, language, 
especially grammar skills. Most teachers believe that ITAs who have good 
communication skills could pass the test. However, even if the proficiency level of 
the speaker is high, s/he might hesitate to talk and not provide enough content. 

Baling the test: 

Grading the test seems to be very tedious. The descriptors are hard to learn 
quickly. The descriptors are useful in rating the grammatical competence but not 
sociolinguistic or strategic competence, since there is no interaction involved as 
testees speak on a tape. We do not know how the language may be used in the 
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classroom. They do not show how an ITA will interact with undergraduates or 
whether s/he can successfully convey information. So there is both a cultural and 
linguistic gap. It is hard to use the descriptors for rating accurately with a 
language group that the rater is familiar with. If a rater is used to hearing an 
accent, it becomes more comprehensible for that rater. That accent is easier for the 
rater to understand, so s/he could grade those students with familiar accents higher 
than one should or rate others lower than they should be graded. Personal biases 
or past experiences may afi-ct the grading. It is hard to find sufficient interrater 
reliability. What is comprehensible to some could be very different from what other 
persons think of as comprehensible. 

Some teachers do not agree with some of the descriptors on the test. It is hard 
to go along with a set of rating scales that one cannot agree with. Students can get 
points simply for responding, even if the responses are ungrammatical. As long 
as they are saying something thev get points for content or pronunciation. And 
even if their answer has nothing to do with the question that is being asked, simply 
because they do not understand the question, they will get points. And if that is not 
made clear to ITAs before they take the test, that person may remain silent. The 
test should make clear that testees should respond no matter what because they 
are getting points for anything they say. 

Comprehensibility is always the difficult category. There is no clear difference 
between comprehensibility and intelligibility. If the pronunciation is poor, does that 
necessarily mean comprehensibility is poor? Can comprehensibility as high as 
the highest score in the section where comprehensibility is evaluated? 

The most frequently-mentioned positive aspects of this test indicated by most 
teachers is that they are pleased to have a standardized test that does not require a 
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lot of administration time, as opposed to, most other oral interview-style 
assessments (like the ACTFL Oral Proficiency Interview) require.^. The variety of 
tasks also gives students a chance to show their area of strength, because some 
studer.is have strengths in small-detail-level oral English (sentence level, grammar, 
pronunciation), and others have more strengths in more context-oriented material. 

Conclusion: 

This study revealed that there is no overall significant difference in ITAs" 
affective reactions to the SPEAK test in terms of ethnic backgrounds or their fields 
(Math-social science); nor there is any significant difference between ITAs who took 
the test in 1 988-89 and in 1 989-90. ITAs' perceptions of the test do not differ 
when compared by sex, age, or formal years of English. However, people who 
passed the test and others who did seem to have differing opinions about the test. 
Some instructors mentioned the lack of face validity and voiced some concerns 
about how difficult it was to judge the overall comprehensibility of the ITA. In a 
testing situation where individuals' speaking proficiency is tested it is inevitable to 
think of mixed or combined testing situations, using both SPEAK Test types of 
questions as well as more realistic oral interviews or teaching samples. 
When ITAs perform in another language, the other language that they speak 
portrays their social identity. However, if ITAs do not accept the social identity that 
is forced through the SPEAK test, they find themselves speaking "nonsense". In 
terms of policy decisions, a test can be considered as a social criterion that can put 
people in different "tracks" and "classifications". Thus, using a single educational 
test can put the individuals at nsk. The SPEAK test situation also shows how much 
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power and control the system gives to a single test , and deterniny the "truth" 
about oral proficiency of an individual. 
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THE SrEAK TEST 



This questionnaire is part of a research project that is being carried out by a 
graduate student at Curriculum and Instruction in Second Languages and 
Cultures Education at the University of Minnesota. The results of this project 
will be used to assess the reliability and accuracy of the oral test and to 
compare the accuracy with other oral tests that are used for international 
teaching assistants at different universities in the U.S. For this reasons it is 
very important that your responses be as frank as possible. 

It is important for you to know that the opinions you give are confidential 
and will not interfere in any way with your relationship to the department 
that you are in. 

Name Country: Date: 

Phone: 

signature: Date of the SPEAK Test: 

Directions: Please place an "X" on the agreement and disagreement scale 
which best expresses your opinion. Note that you are ^ked to explain you 
response. 

e.g. disagree X : : : agree 

Questions 

Department: Age Sex 

Please give answers to the following questions. 

* How many months have you been in the U.S. or other English-speaking 
countries? 

* How many weeks have you studied English in the U.S.? 

* Are you currently taking English courses? 

* How many years have you studied English in your native country? 

* Have your ever taught a course using English before? 
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Which is the highest academic degree that you have received? (Circle) 

a) Bachelor's degree 

b) Master's degree 

c) Ph. D. degree 
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* How many years of teaching experience do you have? 

* Which of the following are you planning to do/ or are you doing at this 
university? 

a) grading exams only 

b) grading exams and term-papers 

c) proctoring exams only 

d) holding office hours only 

e) doing tutorials only 

0 setting up lab equipment only 

f) setting up lab equipment and leading lab sessions 

g) teaching a class (recitation hours are counted as teaching) and having 
office hours 

h) having the full responsibilities for a class 

* How many quarters /years have you been teaching at the University? 

* Which of the English tests that you have taken is an accurate evaluation 
of your English in general? (Circle as many as apply) Please indicate when 
you took the test. ( e.g. 1985) 

year 

a) TOEFL 

b) Minnesota Battery 

c) Test of Spoken English 

d) The SPEAK test 

e) other (please indicate) 

1. I believe the SPEAK test is an accurate evaluation of my ability to speak 
English. 

disagree : : : agree 

Please explain why: 



2. I believe that the SPEAK test is an accurate evaluation of my teaching 
ability in English in a class. 



disagree : : : agree 

Please explain why: 



3. Which part of the SPEAK test did you enjoy doing the best? (There are 
seven parts in the SPEAK test) Circle as many as apply. 

I- Section one (In this section there are some questions about yourselO 

II- Section two (In this section you are asked to read a paragraph) 

III- Section three ( In this section you are asked to complete sentences ) 

IV- Section four ( In this section you see a series of pictures that tell a 

continoues story; you are asked to tell the story that the pictures show) 

V- Section five ( In this section you look at a picture and are asked some 
questions about the picture) 

VI- Section six ( In this section you are asked to give your opinions on topics 

of international interest and to describe certain objects) 

VII- Section seven ( In this section you are asked to explain a class schedule 
or a notice) 



Explain why: 



4. On the SPEAK test you are given scores for your pronunciation, your use 
of grammar, fluency, and lastly for your overall comprehensibility ( how 
understandable your English is). Could you give an estimated score that you 
got on the SPEAK test? 

The score for overall comprehensibility is reported on a scale that ranges 
from 

low 0 100 150 200 250 300 completely comprehensible 

For pronuncaiton, grammar, and fluency the score range is (0 to 3) 
(very low ) 0 0.5 1.5 2.5 3.0 (almost like a native speaker) 



Pronuncitation Grammar Fluency COMPREHENSIBrLITY 

(your total score on the test) 



(score) 



(score) (score) 



(score) 
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I'll let the TA English Program tell the researcher my score on the SPEAK test, 
signature 



5. I felt nervous before the test 

disagree : : : agree 

Please explain why: 



6. The test handbook that I read before taking the test was very helpful. 

disagree : : : agree 

Explain why: 



7. I think there should be a week long preparation program for the test before 
students take the test. 

disagree : : : agree 

Explain why: 



8. Talking to the tape made me very nervous during the SPEAK test. 

disagree : : : agree 

Explain why: 



erJc hi 



9. I had had 
disagree 
Explain why: 



a lot of information about the test before I took the SPEAK test, 
: : : agree 
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10. I wish I had my dictionary with me during the SPEAK test. 

disagree : : : agree 

explain why: 



11. The SPEAK test was too short. 

disagree : : : agree 

explain why: 



12. At the end of the test I felt as if I had been tested for my oral proficiency. 

disagree : : : agree 

explain why: 



13, At the end of the test I was content with the answers that I had given for 
each question. 

disagree : : : agree 



explain why: 



14. rd rather have a conversation with one of the testers than take the 
SPEAK test. 

disagree : : : agree 

explain why: 



15. I believe that the SPEAK test is an ideal way of testing 

international teaching assistants' teaching and speaking skills. 

disagree : : : agree 

explain why: 



16. I believe that American teaching assistants should also take the SPEAK 
test. 

disagree : : : agree 

explain why: 

17. At the end of the test I felt that I had had an adequate opportunity to 
demonstrate my ability to speak and teach in English. 

disagree : : : agree 

explain w^hy: 



18. I think international teaching assistants should take more than one test 
before they are assigned to be teaching assistants. 

disagree : : : agree 

explain why: 



19. I believe that international teaching assistants who have a lot of 
experience in teaching in their countries should start teaching at 
the University of Minnesota without being tested. 

disagree : : : agree 

explain why: 



20. If I had taken the same SPEAK test on another day the results would be 
different. 

disagree : : : agree 

explain why: 



21. I believe that oral tests are necessary for international teaching 
assistants. 

disagree : : : agree 

explain why: 



22. I liked the SPEAK test. It is an ideal test for international teaching 
assistants. 

disagree : : : agree 

explain why: 



'^23. I thought the SPEAK test tested what I learned in the TA 
communication skills class this quarter. (This question is not for everyone). 

disagree : : : agree 

explain why: 



24 I wanted to take the SPEAK test again after I took a quarter long classroom 
communication skills course. 

disagree : : : agree 

explain why: 



25. It was difficult to understand the speaker on the SPEAK test because of 
her/his accent. 

disagree : : : agree 



explain why: 



26. I believe if 1 had been given several different kinds of tests the results of 
those tests would have told the TA English program more about my English 
speaking proficiency. 
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disagree : : : agree 

explain why: 



27. I feel more comfortable when I take an oral exam in a group, or with 
another student than taking the test by myself, talking to a tape. 



disagree : : : agree 

explain why: 



28. I wished I had taken an oral test that would test me on the subject matter 
in which that I plan to do my graduate study. ( e.g. physics, math, food 
science, engineering, sociology, psychology, literature, etc.) 

disagree : : : agree 

explain why: 



29. Instead of taking the SPEAK test, I wished one of the professors in my 
department had interviewed me about my general competence in the subject 
matter that I am going to teach at the University of Minnesota. 

disagree : : : agree 

explain why: 



30. After taking the SPEAK test and /or taking the TA communication skills 
class I feel more secure to teach my subject to native speakers. 



disagree : : : agree 

Explain why: 



31. During the SPEAK test I wish I had been asked questions like the 
following. Please list them here. 



32. For ITAs who use languages other than English in the classroom; do you 
feel you should be expected to take the Speak test? 



Other comments: 



Thank you for your time and cooperation. 



